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SUMMARY 


A method is presented for implementing the state estimator of the manual 
control model when the system output is a visual display of arbitrary form; 
that is, the display may be pictorial, including real world, or made up of 
dials and pointers. The method is used to provide error criteria for a look- 
point controller that appears to be capable of modeling human scanning 
behavior. This model, if combined with a model of the control process, should 
be useful in predicting effects of changes in displays on performance of flight 
tasks . 


INTRODUCTION 


One of the most important elements of a model of manual control is some 
form of state estimator. This element receives system outputs and converts 
them into an estimate of the system state in a form suitable for input to a 
control algorithm. 

The state estimator is usually modelled as a Kalman estimator, which 
minimizes the variance of the estimated state, and which is capable of 
accepting sampled data. The output of the estimator includes an estimate of 
the covariance of the state estimate. This covariance depends on the probable 
errors in the data, and not on the data. In many cases, such an estimate of 
covariance is sufficient for an analysis of system performance by the use of 
covariance propagation techniques. Furthermore, when covariance can be 
estimated prior to actual data input, an optimum sequence of samples can be 
predetermined. 

Due to the nature of the visual sense, human observers are usually forced 
to scan a scene in a series of lookpoints in order to extract its information 
content. If the scene is changing, each lookpoint constitutes a sample of 
output data of a dynamic system. All that is required for applying the Kalman 
estimator as a model of human visual observation is a means of estimating 
probable errors of observation at any lookpoint. 

This paper presents a means of estimating the errors to be expected when 
a human observer estimates the state of a system by looking at a display of 
some set of system outputs. The display may be pictorial, including "real 
world," or made up of discrete dials, pointers, etc. 
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First, the method of estimating the covariance of the state estimate at a 
given lookpoint will he described. Then the means, and some results, of 
devising a lookpoint controller to simulate human scanning behavior will be 
discussed. This latter work is presented in full detail in reference 1. 


SYMBOLS 


x 

/\ 

X 

y 

[C] 

V 
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[cov(y) ] 
[cov(x) ] 

0 

(t> 

Abbreviation: 


state vector 
estimated state vector 
output vector 

matrix of coefficients relating state and output vectors 
standard error of observation of output 
transpose of C 

covariance matrix of output vector 
covariance matrix of estimated state vector 
pitch angle 
roll angle 
yaw angle 


VSI Vertical Speed Indicator 

GSI Glide Slope Indicator 

A dot over a variable denotes a derivative with respect to time. 


METHOD OF ANALYSIS 


Figure 1 is a manual control block diagram that is borrowed from 
reference 2. The state estimator in figure 1 will be considered to have in it 
some means of solving the relationship: 


(i) 


where 
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y(t) = c x ( t ) + v(t ) 

x(t) is system state vector 
C is a matrix of constant sensitivities 



y(t) is visible system output vector 
V(t) is random error vector 

The sources of error V(t) are in the display and measurements V (t) 
and in the visual sensing of the pilot V^(t). 

V(t) = v m (t) + v d (t) 

The visual errors are the ones that vary with lookpoint. Thus, figure 1 

may be considered to represent the pilot model at a fixed lookpoint. At a 
different lookpoint, , and therefore y and x, may be different. 

The perceived system output y(t), which is the input to the 
state estimation process, might be specified in several ways. For example, 
in a display composed of discrete instruments, one of the elements of y(t) 
might be taken to be the altitude, since that quantity is displayed by the 
altimeter. A more general approach that allows treatment of pictorial as well 
as discrete displays is to take as elements of y(t) the displacement, rate 
of change of displacement, and rate of change of displacement of points in the 
display (e.g., points on the altimeter needle). Consider the display to be 
broken into segments. The portion of the display in each segment is rigid so 
that one point in a segment may be taken to represent the whole segment. 
Rotating display elements should be represented by more than one segment, so 
that rotation of any one segment can be neglected. This point may be seen to 
move vertically or horizontally or both in response to one or more of the state 
variables x(t). Taking each component as a separate indicator of system out- 
put, each point in the display may provide four elements of y(t): vertical 

and horizontal displacement and vertical and horizontal rate of change of 
displacement from a nominal position. 

The next step is to perturb each state variable in turn and to calculate 
the resulting effects on each display segment. These effects will be expressed 
as linear influence coefficients which are, either exactly or approximately, 
partial derivatives of vertical and horizontal angular displacement and rate 
of displacement, as measured at the observer's eye, with respect to the system 
state variables. These influence coefficients are, of course, the elements of 
the matrix C in equation 1. 

The random error V(t) remains to be specified. That part of it that is 
due to the visual sense V^(t) depends on the observer's acuity at each dis- 
play point, which in turn depends on the location of the display point with 
respect to the observer's lookpoint. From a knowledge of the observer's 
resolving power at any point in his visual field and of where he is looking^ 
one can estimate the element of V(t) for each display segment. Typical 
resolution curves are shown in figures 2 and 3. For any given lookpoint, the 
eccentricity angle (visual angle between lookpoint and display point) of each 
display point is calculated, and the corresponding resolutions from figures 2 
and 3 are taken as standard deviations for calculating V^(t). 
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Now the minimum variance estimate of x(t) may be formulated from 
equation 1. This estimate is: 

x(t) = [c x [cov(y)]- 1 c]"V [cov(y)]" 1 y(t) (2) 

The matrix cov(y) is the covariance of y(t) which, for the case of 
uncorrelated measurement errors, is a diagonal matrix formed by squaring the 
elements of V(t). If measurement errors are known to be correlated, there 
will of course be off-diagonal terms in cov(y), but equation 2 is still valid. 
What is required in this paper, as well as :x(t), is an estimate of its 
covariance. It may be shown (ref. 3) that the covariance of x(t) is: 

cov (x) - jc T [cov(y )] 1 cj 1 (3) 

It is seen that, since for each different lookpoint there is a different 
matrix [Cov(y)] 9 the covariance of the estimated state varies with lookpoint. 


Example 


In order to demonstrate how to calculate cov(x) for a given lookpoint, a 
simple example has been concocted. Two hypothetical displays will be compared 
for each of two lookpoint s. 

Figure k shows two displays, each of which is capable of showing three 
variables. When all three variables are at zero, both displays look the same. 
In Display A all three line segments move together as in figure Mb). In 
Display B, each segment responds to a different variable, as in figure Me) . 
Suppose that the variables presented are pitch, roll and yaw angles, and let 
the display be viewed from such a distance that movement of 1 degree visual 
angle represents 1 degree pitch or yaw (according to direction), and 1 degree 
rotation of the display in its plane represents 1 degree of roll. 

The displays are of such size that each line segment subtend 11 degrees 
of visual angle. 

The display area must be divided into discrete areas, a point in each area 
being taken as the indicator of system output for that area as shown in figure 
5 . Sensitivities of these points to changes in state variables are calculated 
in terms of change of visual angle or angular rate per unit change in each 
state variable. These calculated sensitivities are given in table I. The 
sensitivity matrix [C] would have in the general case four rows for each 
segment* For the illustrative cases in this paper, certain movements were 
considered negligible or not visible. The sensitivities of these movements, 
being zero, were omitted from the table to save space. For example, horizontal 
movement, due to rotation, of a point on a horizontal line was neglected. 

Also, motion along a line was considered not visible. If the lines were really 
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made up of dots, this motion could be seen, and the sensitivity matrix would 
have corresponding terms. Segment 6 is, in fact, the only one in which both 
vertical and horizontal components of motion were considered to be visible, 
and so it appears four times in table I for each display. 

The accuracy of observation depends on the observer's acuity and his 
lookpoint. In order to simplify estimation of parafoveal acuity, it is assumed 
that contours of constant acuity are circles centered on the lookpoint. This 
assumption is not essential to the method, but the spread of available acuity 
data is great enough that a more detailed mapping seems unwarranted. 

Two lookpoints were chosen for illustrating the method: one at the 

intersection of the lines, and one 6 degrees to the right of the intersection, 
on the horizontal line. For each lookpoint, eccentricities were calculated 
for the points in the display for which sensitivities were calculated. For 
these eccentricities, resolutions were read from the curves. The resolutions 
were used as the elements of [cov(y)]. 

The covariance matrices of the estimated state cov(x) were computed from 
equation 3 and presented in tables II and III. Table II is for Display A. 

Note the way correlation between pitch, roll, and yaw, as shown by the off- 
diagonal terms, changes with lookpoint. There is no correlation between 
displacement and rate for either lookpoint. 

The covariances for Display B are diagonal matrices for both lookpoints. 

As might be expected, the variance of estimate of any given state variable 
depends on lookpoint. This dependence on lookpoint is much less for those 
variables that are perceived through sensation of rate, as rate resolution 
varies much less across the field of vision than does position resolution. 


Scanning Behavior 


If the display elements did not move while the observer looked around, the 
covariance of x(t) could be reduced by combining directly the information from 
several lookpoints. This reduction would be easy to estimate. However, since 
x(t) represents the state of a dynamic system, the observer, in trying to 
improve his estimate of any state variable by attending to another point in the 
display, finds that the uncertainty of the information he obtained from the 
first point increases while he looks at the second. The optimum means of 
combining sequential observations of a dynamic system is the Kalman estimator. 
This estimation algorithm also provides a method for deciding which one of a 
number of possible observations it would be best to make, provided that the 
probable error of each possible observation is known beforehand. Combining 
the Kalman estimator with the method of this paper, for estimating the covari- 
ance matrix of the output, cov(y), one may devise a lookpoint controller, as 
has been done in reference 1, from which the following material is taken. 
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Figure 6 shows the information flow for this controller, which has been 
applied to the instrument array shown in figure 7. Figure 8 shows what happens 
to the variances when the lookpoint is arbitrarily forced to follow the time 
history in the figure. For these results, the system state transition matrix 
that governs the growth of covariance is taken to be that of a second order 
dynamic system without damping or cross-coupling. 

In order for the lookpoint controller to choose its own lookpoint, it 
must have some strategy. Figure 9 illustrates what happens when the lookpoint 
is chosen so as to provide information about the state variable with the 
greatest weighted variance. (it is necessary to weight the variances because 
of dimensionality of each state variable and its importance in control of the 
system.) Figure 9 represents a case of an autopilot-controlled ("coupled") 
landing approach where the command bars in the flight director are inactive. 
Figure 10 is the computed time history of lookpoints over 6.5 seconds, when 
the GSI and artificial horizon are combined into a single lookpoint identified 
as Flight Director. 

A manually controlled landing approach was simulated simply by adding 
command bars as variables that need attention. The same lookpoint selection 
strategy was used, with results as shown in figure 11 when command bars are 
included along with GSI and artificial horizon in the Flight Director. 

In spite of the many simplifying assumptions, the time histories in 
figures 10 and 11 are quite "Humanoid." The flight director gets most of the 
attention, and it gets more attention in manual control than in monitoring the 
autopilot (68 percent of total time compared to 57 percent in monitoring), 
During monitoring, transitions between peripheral instruments are more likely 
to happen than during manual control, where nearly all transitions are between 
flight director and peripheral instruments. However, because of the assump- 
tions and especially because a number of instruments were omitted (Horizontal 
Situation Indicator, for example), there is no direct comparison with available 
eye movement data. 


CONCLUDING REMARKS 


A method has been presented for implementing the state estimator of the 
manual control model when the system output is a visual display of arbitrary 
form; that is, the display may be pictorial, including real world, or made up 
of dials and pointers. The method has been used to provide error criteria for 
a lookpoint controller that appears to be capable of modeling human scanning 
behavior. This model, if combined with a model of the control process, should 
be useful in predicting effects of changes in displays on performance of 
flight tasks. 
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TABLE I. - SENSITIVITY COEFFICIENTS 
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TABLE I.- Concluded 
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TABLE II.- DISPLAY A COVARIANCES 
Units are (arc min)^ and (arc min/sec)^ 
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Figure 1.- Manual control system block diagram. 










(a) Display formats A and B at rest. 



(c) Display B deflected in pitch, roll, and yaw. 
Figure 4.- Fictitious displays used in illustrative example 




STATIC SYSTEM INFORMATION PROCESSOR 



Figure 6.- Information flow for dynamic system. 
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Figure 8.- Fixed lookpoints. 
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Figure 9.- Lookpoint controller minimizing maximum weighted variance 

in automatic landing approach. 
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Figure 10.- Time history of lookpoints predicted by model for "coupled" landing approach. 
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Figure 11.- Time history of lookpoints predicted by model for manual landing approach. 



